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^SJ ' In order to study certain questions concerning the distribution 

of the overlap in Sherrington-Kirkpatrick type models, such as the 
chaos and ultrametricity problems, it seems natural to study the free 
energy of multiple systems with constrained overlaps. One can write 

Mh \ analogues of Guerra's replica symmetry breaking bound for such sys- 

(~| ■ terns but it is not at all obvious how to choose informative func- 

tional order parameters in these bounds. We were able to make some 
progress for spherical pure p-spin SK models where many computa- 
tions can be made explicitly. For pure 2-spin model we prove ultra- 
metricity and chaos in an external field. For the pure p-spin model for 

»vj . even p > 4 without an external field we describe two possible values 

^ ' of the overlap of two systems at different temperatures. We also prove 

^Nj I a somewhat unexpected result which shows that in the 2-spin model 

00 ' the support of the joint overlap distribution is not always witnessed 

^^ ' at the level of the free energy and, for example, ultrametricity holds 

xj , only in a weak sense. 

o 

^D ' 1. Introduction and main results. Let us consider a Gaussian-Hamiltonian 

c~| ■ (process) Hj\i{a) indexed by cr G M witli covariance tliat satisfies 

-(— > 



(1.1) 



^EHn{<t')Hn{(t^) - ^{Rl,2) 



< CN, 



- ,— , , wliere cat — > 0, i?i,2 = N^^ J2i<N '^i'^f is tlie overlap of configurations <T^,a^ 

^ ' and ^ is a smooth enough convex even function with ^(0) = 0. We define 

j^ ■ d{x) = xS,'{x) — S,{x). Even though we will state some basic results for a gen- 

eral function ^, our main results will deal with the pure p-spin SK Hamilto- 
nians that correspond to ^{q) = q^/p for even p>2. For example, one can 
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2 D. PANCHENKO AND M. TALAGRAND 

consider 

(1.2) HNicT)= ^^(p„i)/2 E 9^l,...,^p^n•••^ip, 

y^ i<ii,...,ip<N 

where {gi^^...^ip) are i.i.d. Gaussian random variables. For p = 2 this is a 
classical SK Hamiltonian [5]. The factor p~^''^ is not important and is chosen 
so that ^'{q) = q^~^. In this paper we will consider the spherical model when 
the spin configurations cr belong to a sphere Sn of radius \/]V and the a 
priori distribution of cr is the uniform measure A at on Sjsf. Given an inverse 
temperature /3 > and an external field /i 6 M, the "free energy" is defined 

by 

FN{P,h) = ^Elog / eMf3HN{(T) + hJ2 ^^] ^Ajv(cr). 

Its limit limAr^oo-^Af(/3, ^) ='P{l3,h) was computed in [1] and the compu- 
tation was made rigorous in [8]. The limit V{P,h) is the analogue of the 
Parisi formula in the Ising SK model [6, 7]. For the spherical model it is 
somewhat simplified by the fact that certain computations become more ex- 
plicit. Namely, if, given k>l and given two sequences m = (m;)o<z<A; and 
q = {qi)o<i<k+i such that 

= ITlQ < nil < • • • < TTT-k = 1) 

q = qo<qi<---< qk+i = i, 

and given a parameter 6 > 1 , we define for I < k 

di= Y^ mp{C'iqp+i) - ^'{qp)) and Di=h-di, 

l<p<k 

then 

ViP,h)= M Ub-1-logb+^ih^ + eiqi)) 

o,fe,m,q Z \ U\ 

(1-3) 



+ E ;^i°g%^- E m{e[qi+,)-e{qi))\. 

V{P,h) is also given by the Crisanti-Sommers representation (see Section 4 
in [8]) as follows. If 6i = 'Ei<p<k'^piQp+i - %) then 



(1.4) 



V{(5,h)=mi]-(hHi + ^qi+ Y. — logT^ + log<5fc 

+ Y. mmqi+l)-i{qi))\ 

KKk / 
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When h = we will write V{P) := V{P, 0). The goal of this paper is to prove 
and analyze some bounds on the free energy of multiple copies of the system, 
possibly at different temperatures, coupled by constraining their overlap. 
Let Q he a n X n symmetric nonnegative definite matrix with elements 
Qjj' S [—1, 1] and Qjj = 1. Given e > consider a set 

(1.5) Qe = {{ct\ . . . , cr") g S]^ : Rjj, G [qjj, - e, Qjj' + e] for j,j' < n} 

and given /3i, . . . , /3„ > and /ii, . . . , /i„ G R we define a "free energy" of the 
n-configuration system constrained to the set Qs by 

(1.6) Fr,iQs) = ^K\ogf exp( ^/3,i?jv(^^) + E ^J' E ^H ^^^- 

"'Qe \j<n j<n i<N ) 

Obviously, -F/v((5e) < J2j<n^Ni(3j, hj) and as a result a trivial bound would 
be 

(1.7) limsupF^(Q,) <Y.V{(3j,hj). 

We would like to construct some nontrivial bounds on Fis[{Qe) that would 
yield some information on the support of the distribution of the overlaps 
{Rj.j') under the product Gibbs measure by showing, for example, that for 
some constraints Q 

limsuplimsupF/v(<5e) < /^ V{(3j,hj), 

which by concentration of measure would imply that with high probability 
the overlaps cannot be in configuration Q for the product Gibbs measure. At 
this moment, the only approach we could conceive for proving such bounds 
is based on an analogue of Guerra's interpolation [3] that was used in [7] for 
two coupled systems at equal temperatures and external fields, and explained 
in more generality in [9]. It was also explained in [9] that it seems to be not 
obvious at all how to choose parameters in these bounds that would at least 
witness the obvious inequality (1.7). In this paper we will describe several 
situations when we were able to find such parameters. The methods of the 
proofs are at least as interesting as the results they imply since they shed 
some light on the difficulties of finding informative parameters in the bounds 
and give some hope that, in principle, these bounds might be "correct" and it 
could be only a (very difficult) technical problem to find suitable parameters 
in more general situations. 



4 D. PANCHENKO AND M. TALAGRAND 

The pure 2- spin SK model. The first case we will consider is the pure 2- 
spin SK model in (1.2) with p = 2 without external field, that is, /i = 0. What 
makes this case particularly simple is that due to the proof of Proposition 2.2 
in [8] the infimum in (1.4) [and (1.3)] is achieved on the replica-symmetric 
choice of parameters, that is, for A; = 1, so that (1.4) becomes 

V{(3) = inf \ {-^ + log(l -q)+ (3\{l) - [3\{q) 
ge[0,l] l\l — q 

where ^{q) = q^ /2. It is easy to check that the infimum is achieved on g = 
when (3 <\ and q = \ — 1/ (3 when /3 > 1. The first case is trivial in many 
respects, so we will only look at the second case /3 > 1 for which the free 
energy above becomes 

(1.8) V{(i) = \{2(5-l-\og(5). 

We will prove the following bound on Fn[Q^) in (1.6) when the external 
fields hj = 0,j < n. Given a matrix of overlap constraints Q = {qjj') let us 
define a matrix 



(1.9) Q = {^PjPj'Qj,f) 

and let ri, . . . ,rn be its eigenvalues. Consider the function 



(1.10) f{r 

2' 



logr + ^r'^, for < r < 1, 

2r-i for Kr, 



and note that for < r < 1 , 



(1.11) /(r)=logr + ir^<2r ^ 



The following theorem holds. 

Theorem 1. For any matrix of overlap constraints Q we have 
(1.12) limsuplimsupF7v(Q,) < i ^(/(r^) - log/3j), 

where {rj)j<n o-re the eigenvalues of ('i-'d) and f{r) is defined in (^l.lOj. 
The right-hand side of (^1.12^ is strictly less than J2i<n^i(^j) ^f the smallest 
eigenvalue minrj is <1. 

The second statement of the theorem follows from (1.11). Indeed, note 
that if all eigenvalues Vj of Q satisfy rj > 1 then the bound (1.12) becomes 

E ^(2^i - i - i°g/?.) = E hm - i - log/?.) = E np,) 

j<n i<'« i<'" 

by (1.8) and since EjKn^j = Tr(Q) = EjKnf^j- By (1.11), the bound (1.12) 
will be strictly less than J2j<n'^i(^j) if the smallest eigenvalue minrj < 1. 
Let us look at some consequences of this. 
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Example 1. Let n = 2, /3i,/32 > 1- For Ri^2 ~nG [-1,1], 

and 

It is easy to check that r2 > 1 if and only if 



1 



and Theorem 1 (together with standard concentration of measure) imphes 
that the overlap of the coupled system can not exceed ^qiq2- In the case of 
two equal temperatures Pi = (32 = P this means that the absolute value of the 
overlap can not exceed q = l — 1/(3, which could also be obtained by methods 
of [8], "breaking" replica symmetric choice of parameters. In Lemma 2 below 
we will give a general statement that says that if for j <2 the overlap of the 
system j does not exceed qj than the overlap of a coupled system does not 
exceed ^Jq\q2- Thus, Example 1 could be obtained without the application 
of Theorem 1. However, (1.12) provides an explicit constructive bound. 

Our next example will be less trivial, but before we proceed let us make 
one observation. Using the fact that for /? > 1, /i = 0, 

lim F7v(/3):= lim — Elog / exp/3if7v(cr) dAAr(cr) = P(/3) 
given by (1.8) and since both Fn{(3) and V{(3) are convex in (3, we have 
hm F'^{(3) = lim \(3{l -¥.{^2)) ='P' il^) = 1 " :^' 

iV— »oo N^oo Z Zp 

where (•) denotes the Gibbs average. This implies that 

lim E(i22 ) = [1 _ 1 ") =^2^ 

Example 1 for two equal temperatures implies that for any e > 0, 

lim E(/{|i2i,2|>5 + e}) = 0. 

A— >oo 

These observations combined, of course, imply that for any e > 0, 
(1.14) limE(/{|i?2 g2|>^|^^0^ 

A— >oo 

that is, the overlap can take only values close to itg. 
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Example 2 ( Ultrametricity) . Let us consider three copies of the system 
(n = 3) with the same /? > 1. Ultrametricity means that with high probabihty 
in the disorder (randomness of the Hamiltonian H^)^ for any e > 0, the 
Gibbs measure of the event 

i?2,3 > min(i2i,2, -^1,3) - ^ 

is close to one. By (1.14), the overlaps Rjj',j // can only take values ±q 
and, thus, the only possible nonultrametric overlap configuration is described 
by the constraint matrix 

/I q q 
Q=[q 1 -q 
\q -q 1 

It is easy to check that for this matrix 

ri=r2 = /3(l + g) and r3 = /3(l-2g). 

First of all, the matrix Q is positive definite only if 1 — 2g > 0, that is, /? < 2. 
Also, for /3 > 1 we have r^ = /3(1 — 2q) = 2 — (3 <1 and, therefore, such a 
configuration Q is not in the support of the Gibbs measure and we have 
ultrametricity. Some intuition for the fact that the overlap takes two values 
ibg is that the Gibbs measure is dominated by two symmetric "states" such 
that the typical overlap of two spin configurations within each state is equal 
to q and the overlap of spin configurations from different states is —q. Of 
course, such a picture naturally excludes the overlap configuration given by 
Q above, but a rigorous proof is another matter. 

Let us now go back to the Example 1 for (3i = [32 = /?• For the value of the 
overlap Ri^2 ~ u such that \u\ < q, Theorem 1 provides only a trivial bound 
2V{(3) of the type (1.7) while, on the other hand, (1.14) proves that the over- 
lap cannot take values between —q and q. At the level of large deviations 
the bound (1.12) does not detect this and one might ask whether (1.12) is 
simply not sharp in this case. A similar question may be asked about Exam- 
ple 2 which proves ultrametricity only in the weak sense since due to (1-14) 
we only had to consider one nonultrametric configuration. However, there 
are nonultrametric configurations Q = {qj,j'), that is, (72,3 < 111111(^1,25 91,3)) 
for which Theorem 1 will only give a trivial bound 3V{P). Again, could 
it be that (1.12) is simply not sharp in that case? The answer to both of 
these questions is negative as shown by the following theorem. This result 
is surprising because it shows that in this model the ultrametricity (and 
the chaos) cannot be proved at the level of large deviations and, therefore, 
it is possible that in other models, for example, in the Ising SK model, a 
similar situation occurs and one should be cautious in one's efforts to prove 
ultrametricity at the level of free energy. 
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Theorem 2. In the notation, of Theorem 1, if (3j = [3 > l,j <n and 
minrj > 1 then 

(1.15) lim lim Fjv(Qe) = ?^P(/3), 

that is, in this case the bound (1-12) is sharp. 

The proof of this theorem is an extension of the methodology in [7]. It 
rehes on certain a priori estimates, Theorem 6 below, which generally become 
much more difficult to prove for multiple copies of the system compared to 
a single system. For example, we do not know how to do this for the Ising 
SK model or even for the spherical p-spin model for even p > 2. In the 
setting of Theorem 2 we were able to prove these estimates using a special 
"diagonalization" trick developed in Theorem 1. 

Finally, we will prove some facts about the overlap of two pure 2-spin 
systems in the presence of external fields. If hj = 0, we will assume that 
f3j > 1 and let Qj = 1 - l/fij. If hj / 0, let Pj > and let Qj € [0, 1] be a 
unique solution of 

"? + "?* = (T^- 

Then the following theorem holds. 

Theorem 3. Let uq = if hi = and /12 / and let 

_ hih2{l-qi){l-q2) 
''°"l-/3i/32(l-gi)(l-g2) 

if both hi 7^ 0, /i2 7^ 0. Then, for any e > 0, 

lim E(/{|i?i2-no| >e}) = 0. 

Pure p-spin model, for even p > 4, without external field. Next we will 
consider the pure p-spin Hamiltonian (1.2) for even p > 4, which corresponds 
to S,{q) = q'^/p, without external field, that is, /i = 0. It was proven (following 
an argument of [1]) in Proposition 2.2. in [8] that whenever ^"(9)"^ is 
convex, which is the case here, the infimum in (1.3) or (1.4) is achieved for 
k = 2. When /i = one can argue that qi=0 and, thus, the free energy is 



(1.16) 



(?,me[0,l] 2 



+ (l--) log(l -q) + - log(l - g(l - m))) . 
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Proposition 2.3 in [8] states that the infimum wiU be achieved on g = if 
and only if 

(1.17) sup(/32e(s) + log(l - s) + s) < 0. 

The case where the infimum is achieved on q = corresponds to the trivial 
case when the overlap can take only the value zero, so, we will only consider 
the case of (3 large enough, where q ^0- We will prove the following result 
concerning the overlap of two pure p-spin systems. 

Theorem 4. Suppose that f3j,j < 2 are such that the infimum in fl.lOj 
is achieved on qj / 0, that is, (1.17) fails. Then for any e > 0, 

(1.18) lim W{{\Rl,2\>e]r\{\\Rl,2\-^/q^\>e]]) = Q, 
that is, the overlap can take only the values and ±-1/91^2 ■ 

The rest of the paper is organized as follows. In Section 2 we prove a 
general bound on -F/v(Qe) using the analogue of Guerra's interpolation. In 
Section 3 we prove Theorems 1 and 3, in Section 4 we prove Theorem 4 and 
in Section 5 we prove Theorem 2. 

2. Interpolation. In this section we will describe the analogue of Guerra's 
interpolation for the constrained free energy (1.6). Given k>l, consider a 
sequence m = {jni)i<^k such that 

= rriQ < mi < • • • < m^ = 1. 

We may assume strict inequalities since otherwise in (1.3) and (1.4) we can 
simply decrease the value of k. We consider a sequence for < Z < A; + 1 of 
symmetric n x n matrices Q' = (g^- ,/),• ,'<n such that Q^ = and such that 
if we define 

(2.1) Q' = iPjPf^'iq'jj,)) 
then the matrices 

(2.2) A,:=Q'+^-Q' 

are nonnegative definite for < / < A;. Let z/ = {zi,zf, . . . , z") be a Gaussian 
vector with covariance A; and let z; be independent for I < k. Finally, let 
(zi,i)Kn be independent copies of (z/)i<„ for i < N. For < t < 1 we define 
an interpolating Hamiltonian by 

Ht{a\...,cT^) = VtY,PjHN{cr^) 

j<n 

(2.3) 

j<ni<N \ 0<l<k / 
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Let 

(2.4) Xk+i,t{Qe) = log f expHtdX"},, 

where for simplicity of notations we keep the dependence of Ht and A^ on 
{cr^, . . . , cr") iniphcit. Recursively for 1 < / < A; we define 

(2.5) Xi^tiQ,) = — logE,expmzX;+i,i(Qe), 

mi 

where E^ denotes the expectation in (z^ j) for p>l. We define 

(2.6) ifit) = ^EX.^tiQs). 

Clearly f{l) = Fj^^iQs)- From now on TZ will denote a quantity such that 

limsuplimsup|7^| = 0. 

The following holds. 

Theorem 5. We have 

j,j'<n 

(2.7) 

l<l<k j,j'<n 

where the remainder R{t) > 0. 

Proof. The proof of this theorem is a straightforward generalization of 
Guerra's interpolation for a single system [3] and was explained in detail for 
coupled copies in [7]. We will not reproduce it here. D 

Remark {Remainder for k = 1). The remainder R{t) can be written 
explicitly but we will omit its rather complicated definition in the general 
case k >1 since we will only need the exact form of R{t) in the proof of 
Theorem 2 for k = l. If A; = 1, let us define a Hamiltonian 

(2.8) ht{cT\ ...,a-) = VtY^^jH^i^n + E E 4(VT^4, + hj) 

j<n j^ni<N 

and let {■)t denote the Gibbs average on Qe (and its products) with respect 
to this Hamiltonian. Then given two systems of spins (cr^, . . . , <t") S Q^ and 
{p^ , . . . , p") G Qe we have 

(2.9) R{t) = \ J2 PjPrmiR"') - R'^'^'illf) + 0{ql^))t 

j,j'<n 

where R^'^ = X~^ J2i<N ^l Pi is the overlap between a^ and p^ . 
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Theorem 5 provides an upper bound for Fn{Q£) since 

Jo 
and the integral can be bounded using (2.7). Consider a symmetric positive 
definite n x n matrix A = {0'j,j>)j,'j><m define A^+i = A and recursively for 
/ < k define 

(2.10) Ai = A+i - mi^i = A+i - m;(Q'+' - Q')- 

Below we will always assume that A is chosen so that Ai is also positive 
definite. We will denote by \A\ the determinant of A. The following holds. 

Lemma 1. If h.= {hi, . . . ,hn) then for any matrix A and the sequence 
(Ai) defined by (^2.10j, we have 

2(^(0) < Tr(^fc+iQ) - n + {A^^h, h) + Tt{A^^Ao) 
(2.11) 

+ E — log^^-log|Afc+l|+7^. 



Remark. In fact, one can prove that lim£_+olimAr^oo2(/3(0) is equal to 
the infimum of the right-hand side of (2.11) over all choices of A, that is, 
the bound is sharp. We will give a proof of this statement in the replica 
symmetric case k = 1 which will be needed in the proof of Theorem 2, but 
the proof of the general case A; > 1 is different only in that the notation is 
more complicated. 

Proof of Lemma 1. To simplify the notation, for j <n let 

\0<l<k 0<l<k / 

Then, since on the set Qe we have Rjj/ = N~^ (cr^ , a^ ) = qjj/ + 0(e), we 
have 

-^r^k+lfliQe) 



N 



^logj expY.{cT^,z^)dX% 



(2.12) 






llog f^^ exp(j2{a^,zy) -J2a,,,{a^,a^')] dX% + 0{e). 
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We proceed as in Lemma 7.1 in [8]. Let i^a ("■) be a Gaussian measure on 
M^ of density 

Let us write p G R^ as p = scr where cr S Sn so that by rotational invariance 
the law of a under Uaj ^ is Xn and cr and s are independent. Let 7j be the 
law of s. Let us define Cj by 

(2.13) l=jj[{s>cj}) sd-fj{s), 

which, obviously, implies that Cj <1. Therefore, 

(2.14) 7j({s > c,}) > 7,({s > 1}) = ua^^ ({IIpII > Vn}) = exp(-iVTAr(a,-,)) 
and it is easy to check (classical large deviations) that 

(2.15) lim TN{aj,j) = T{ajj) := h{aj,j — 1 — log a,.). 

TV— »oo 

By Jensen's inequality and (2.13), 

j<n ■^iv \j<n j<j' I 

/ exp^(sjcr^z^) 

- ^ ajj'{sja^ ,Sj'<T^ ) dA^ 

exp[Xl(P^'ZJ) 

\j<n 

- H ajj>{p',p^') J dl/ai,! (p^) • • • di^a„,„ip'') 

< f expf^(p'-,Z^) - J2 HAf^^f^')] d,.a,Ap')---diya^Jp''). 

In the last line, all N coordinates are now decoupled. Let i^ be a Gaussian 
measure on M" with covariance Diag(ai^i, . . . ,an^n) and define for i < N 

\l<k Kk ) Kk 



< 

lf]{s,>c 

"^N dllisi) ■ ■ ■ d-JniSn) 

(2.16) 



(M\\p^\\>c,Vn} 
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Then the last hne in (2.16) can be rewritten as expJ2i<N^k+i,i where 



Yk+i,i = exp (x, Zi) - V ( 

"^'■^ J^^ exp ( (x, Zi) - ^ ajj'XjXj, - ^Y. «i.i^i J ^^ 



expYfc+i,i = 1^ exp I (x,Zi) - 22 "-jd'^j^j' ] dv{yi) 



(2vr)". .^- y .^., ~.^^ 



(2.17) 



Combining this with (2.12), (2.14), (2.15) and (2.16) we get 
■ja^^^k+ifiiQe) 

< Y. "i,i'^i,i' + Y "^("ij) + 2 ^"^ T4f" 

jr-<jr-' J<n I I 

(2.18) 

= i(Tr(^Q)-n-log|A|) + l5]i(A-lz„z,)+7^, 

where 7^ = 0(e) + cat for some cat — > 0. This bound will propagate in the 
recursion (2.5) and since Zj are all independent we only need to compute 
what happens with one of the terms (yl~"'^Zi,Zj)/2. Namely, if 

1 1 

yfc+i = n(^~^2;i,Zj), yi = — logEjexprn^yz+i for / < A;, 

2 mi 

then by induction over I it should be obvious that 

(2.19) ip{0) < |(Tr(AQ) - n - log |^|) + Eyi + 7^. 

To compute the sequence (yi) we use the fact that for a Gaussian vector 
g with covariance C, parameter m > and a symmetric positive definite 
matrix A such that A — mC is also positive definite, we have 

— logEexp— (A"^(x + g),x + g) 
m Z 
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which is a simple exercise. Since the covariance of z/^j is A;, using (2.10) we 
get by induction over / < k that 



4(^r'K. + h),zo, + h) + i Y. :;^i°gTix 

and, therefore, 






1 1 . . I, .-^, , . 1 ^ 1 , \Ai+i\ 



Eyi = -Tr(ylr'Ao) + -(^r'h,h) + - ^ —log 

Plugging this back into (2.19) completes the proof. D 

Combining Theorem 5 and Lemma 1 gives an upper bound on F]y{Q^). 
Prom now on we will always set 

(2.20) g'^+^ = Q 

in which case the first sum on the right-hand side of (2.7) disappears and 
the bound becomes 

2FNiQe) < Tr(^fc+iQ) - n + {A^% h) + Tv{A^'Ao) 

(2.21) + E — log^-log|A,.+i| 

- Y, rniY. /5,/5,'(^(g5+l)-e(4.,))+7^. 

l<l<k j,j'<n 

It will often be convenient to think of A^ as a free parameter and think of 

Ak+i as Ak + Afc and write (2.21) as 

2FN{Qe) < Tr(AfcQ) + Tr{AkQ) -n+ {A^^h, h) 

1 - lA+il 



(2.22) +Tr(Ar'Ao)+ J2 — log Mfp " log I ^fc I 

' * m . I Zl . I 

KKfc-l 



mi \Ai\ 



l<l<k j,j'<n 

In general, it is not clear how to choose the parameters in this bound that 
would witness the trivial bound (1.7). In the next section we will show how 
this can be done for the pure 2-spin model. 

3. Pure 2-spin model. 

Proof of Theorem 1. We will first prove Theorem 1 by constructing 
the sequence (Q') and j4 in a special way. Let us mention that in order to 
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prove Theorem 1 it would be enough to consider the bound (2.21) only for 
k = 1. However, to illustrate the general idea we will look at any k>l and 
at the end of our argument it will be clear why for 2-spin model one should 
take k = l. Note that now ^{q) = 6{q) = q^ /2 and i'{q) = q. Let 

and let 

be its Jordan decomposition for some R ~^^ = Diag(ri, . . . ,r„) and orthogo- 
nal matrix O. We now consider a nondecreasing sequence 

= R^<R^<---<R''< ii^'+i 

of diagonal matrices i?' = Diag(r[, . . . ,r^) which means that each sequence 
{fj)i<k+i is nondecreasing and define 



(3.1) Q' = (g;.^.,) = (y^4,) = O'^R'O, 

which also defines Q' and Q\ in particular, 



qjj, = PjPj'qjji = ^JPj/3j'qjJ,. 
Next, for B = Diag(6i, . . . , fe„) where each bj > we let 



(3.2) D = {djj,) = O^BO and A= {^(3jl3j,djj>). 

Then by (2.10) the elements of the matrix Ai are 

V l<p<k / 

\ i<p<k J j,j' 

J~^,(o'^(b- J2 mp{RP+'-Rn)o) . 

V \ l<P<k ) / j,j' 



It is obvious that for any matrix M = (m-jj') we have 






and, therefore, 



(3.3) \Ai\ = \Bi\Y[l3j ioTBi = B- ^ mp{RP+^ - RP). 

j<n l^P^k 
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Similarly, 

1 . I^il 



Tr(Aj;^Ao)= lim —log 



(3.4) ^ 

= lim llog— M-^ = Tr(i?rii?i) = ^^, 

where we denoted by 6j the jth element on the diagonal of Bi. Next, 



Tr(^fc+iQ) = E«.J'9,J' = E Jf^jf^fdj,,"lj,r = TV(DQ^^+i) 



(3.5) 






Finally, 

l<k jj'^n 

(3.6) =1^:^, E ((^S' )' - (4')') 

= ^E-^E((-5"'')'-(-j)')=E-^Ew-r)-^(^^))' 

l<k j<n l<k j<n 

since Q^ = O^ R^O and the Frobenius norm of the matrix is preserved by an 
orthogonal transformation, that is, 

E(4')^ = E(-1)^- 

j,j'<n j<n 

When h = 0, (3.3)-(3.6) result in a bound (2.21) being transformed into 
2FM{Qe)<Tr{BR''+')-n + TT{B^'R')+ Y. ±log%^ 

(3.7) 

-log|i?|-logn/3j- E mJ2(^{r'j^')-^('''j))+'^- 

j<n l<l<k i<n 

Obviously, if we denote the diagonal elements of Bi hy bi, . . . , bl^ so that 

l<p<k 

then (3.7) decouples into the sum of n terms 

ri 1 6'+i 

-i- + 6,rfi-l-log6,-log/3,+ Y — logir 

"j KKk "^' "j 



16 D. PANCHENKO AND M. TALAGRAND 

(3.8) 

- Y: m{e{rf') - 0{r])) 

l<l<k 

which was our main idea and motivation. Note that the same idea does not 
immediately work for p-spin model for p> 2 since (3.6) fails there. We will 
now explain why minimizing this bound it is enough to look only at the 
case k = 1. Besides the explicit term —log(3j, (3.8) is very similar to (1.3) 
for /9 = 1 with minor differences, namely, the range of the sequence (r')Kfc 
is between and r"*" — the jth eigenvalue of the matrix Q'^'^^ — and we 

have bjT- instead of b. Similar to (1.4), one can repeat the proof of the 
Crisanti-Sommers formula in Section 4 in [8] to show that the infimum of 
(3.8) over bj and the sequence {rj)i<k is equal to the infimum of 

1 ^i 

(3-9) ft+ E ^log^ + \og5'^ + Y^mmrY')-i{r]))-\og(3,, 

"j l<Kfc-l"^' '^j l<k 

where (5j = X]Kp<fc "^p(r^ — rp. (This is especially easy to check for k = 1.) 
The proof of Proposition 2.2 in [8] yields that the infimum of (3.9) is achieved 
on the replica symmetric choice of parameters when fc = 1 , 

inf (log(r| - r]) + ^^^ + \{{T^^f - {r]f) - log/3, 

It is easy to check that the infimum is achieved on 

1 f^?-l> ifr2>l, 

and the infimum is equal to /(r?) — log 13 j which proves Theorem 1. D 

Remark. In Theorem 2 we will prove that the bound of Theorem 1 is 
sharp in when f3j = (3 >1 for j <n and when the bound is nV{P). The proof 
of Theorem 1 shows that this happens when all r? > 1 and (3.10) implies 
that r| = r? — 1. Definition (3.1) then gives that 

q2 _ Qi = /3-io^(i?2 - R^)0 = (3^^I 

and 

(3.11) q1 = q2_^-1j and ^i = l3'^{Q^-Q^)=l3I. 

The diagonal elements of Q^ are gj • = 1 — j3~^ = q and off-diagonal element 
are g^ •/ = g^ •/ = g^j/. It is also easy to check that the infimum in (3.8) is 
achieved for bj = 2, that is, B = 21 and definition (3.2) implies that A = 2/3/. 
In fact, instead of checking that the proof of Theorem 1 produces these 
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parameters Q^ and A, one could simply use these parameters in (2.21) and 
see that they result in a bound -F/v((3e) < n'P{(3) +TZ. Clearly, the condition 
minrj > 1 that gives the bound nT'{(3) is equivalent to saying that Q^ = 
Q — f3~^I is nonnegative definite. 

Proof of Theorem 3. We already mentioned that for S,{q) = q'^/2 the 
infimum in (1.4) [and (1.3)] is achieved for A; = 1, 



(3.12) 

+ log(l-g) + /32e(l)-/32e(g)j. 

The critical point condition for q is 

(3.13) h^+p^^'^q)=h^+f3^q = _±_. 

If hj 7^ 0, let qj be the unique solution of (3.13) that corresponds to the 
choice of parameters (3j, hj. If hj = 0, we will assume as before that /3j > 1 
and qj = l- 1/Pj. For k=l, (2.22) is 

2FNiQe) < Tr(yiiQ) + TY(AiQ) - 2 + (A^^h, h) + Tv{Ai^Ao) 

-logl^il- Y. /3,/3,'(^(9,,/)-%,V))+7^. 
ij'<2 

When Ri^2 ~ qi,2 = u for \u\ < yjq\q2-, we will take 

Q^ = f'^^ M and ^i=f"i ^ 
\u q^) V ^ «2 

and the bound becomes 2F]\f{Qir) <U + TZ ior 

U:=^{1- qif + ^(1 - q2? + ai+ a2-2 + 2\u - \og{aia2 - A^) 

(3.14) ^ 

+ ^(a2(/3?gi + /i?) + ai(/3|g2 + hi) - 2(/3i/32U + /ii/i2)A). 

aia2 — A^ 

It is easy to see that for Oj = 1/(1 — qj) = 13 j and A = 

(3.15) \U = V{(3iM)+V{l32M)- 

The derivative of (3.14) in A at A = and Oj = 1/(1 — qj) = (3j is 

IdU 
2~dX 



= u {P1P2U + /ii/i2) = n - (1 - gi)(l - q2)iPiP2U + hih2). 

A=o «ia2 



If both hj = then since /?j = 1/(1 — gj) this derivative is equal to for all 
1^1 ^ ^/qiq2 which means that we can not improve upon (3.15) by fluctuating 
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A around zero. Suppose that hi = and /12 / 0. Then again /3i = 1/(1 — qi) 
and by (3.13), (32 / 1/(1 — q2)- The derivative 

IdU 



2dX 



= u{l-l32{l-q2)) 
A=0 



is equal to zero only if u = uq = 0. Therefore, for \u\ < y/qiq2 and u 7^ 
0, by fluctuating A around zero we can prove that Fn{Q^) < V{(3i,hi) + 
P(/32, /i2) + Tl and by concentration of measure this means that the overlap 
can take only the value between zizy/qiq2- If both hj 7^ then (3.13) implies 
that (3j{l — qj) < 1 and thus the derivative is equal to zero only at one point 

_ /ti/t2(l-gi)(l-g2) 
'"'~ l-(3i(32{l-qi){l-q2) 

and again the overlap can take only the value uq between ±-^(?i(?2- To finish 
the proof of Theorem 3 we need to show that the overlap can not take values 
Kl > \/QiQ2 which will be done in Lemma 2 below. D 

Lemma 2. For j <2 let {■)j denote the Gibbs average of the (random) 
Gibbs measure Gj on Sn (and its products) and let (•)i,2 denote the Gibbs 
average with respect to the product Gibbs measure Gi dSi G2 on Sj^. If for 
any e > 



hm E{I{\Ri^2\>qj+e})j = 

N—>oo 



then for any e > 



lim E(/{|i?i,2|>v/5I^ + e})i2=0- 



Proof. For any integer fc > 1 we can write 

ii,...,ik 



ii,...,ik 
Ji,...,«fc 

\ ii,...,ik ' \ ii,---,ik ' 



/E(i?f_2)iE(i?^_2)2 

and the result is obvious. D 
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Remark. Another approach to proving the above results, as well as 
Theorem 2, for p = 2 model would be diagonalizing the system from the 
beginning by writing the Hamiltonian after orthogonal transformation as 
J2i<N '^i^h where Aj are the eigenvalues of the matrix {gij)ij<N- In [2], a 
large deviation principle was proved for 




and one can similarly prove large deviation principles for several copies of 
the system by including a vector of overlaps. This approach would, probably, 
yield simpler proofs of our results for p = 2 model but in the present paper 
we are trying to develop a methodology that could potentially be used for 
p> 2 models or mixed p-spin models in which case the diagonalization idea 
would not work. In the next section we apply our methodology to the case 
of two copies, n = 2, of the pure p-spin model for p> 2. 

4. Pure p-spin model. In this section we will prove Theorem 4. Unfor- 
tunately, the argument of the previous section does not work because for 
p > 2 equation (3.6) does not hold anymore. Consequently, we were un- 
able to prove a general result for all n > 2. However, for pure p-spin model 
without external field the parameters {m, q) that achieve a minimum in the 
free energy formula (1.16) satisfy certain special properties that will give us 
enough information to find the informative parameters in the bound (2.22) 
for n = 2. These properties are given in Lemma 3 below. 

We assume that h = and in order to avoid trivial situations we consider 
only /3 > such that (1.17) fails. First we will gather important information 
about the parameters {m,q) that achieve the infimum in (1.16). The critical 
point conditions for (m, q) are given by 

,^,',„ = 1( ' _^^). 

mXl — q 1 — + mq J 
(4.1) 

P C{q) = ^log — -, -, — 

m^ V 1~9 / ml — q + qm 

and from now on we assume that {m,q) satisfy (4.1). Let j; > be a unique 
solution of 

(4.2) l = i±^iog(l + x)-i. 

p X'^ X 

It is easy to check that the right-hand side is a convex function decreasing 
from 1/2 to as X increases from to oo and, hence, for p>2 there is, 
indeed, a unique solution x. Define 

fA 'i\ X ^ p-1 x"^ 

(4-3) ^=r—, — , 7 



1 + X p 1 + x 
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Then the fohowing holds. 

Lemma 3. For all (3 such that (1-17) fails we have 
(4.4) qm = x{l-q), mP^^'{q){l - q) = 6, 7n^(3'^e{q) =-f. 

Proof. Introducing the notation A = l — q,B = qm, (4.1) can be rewrit- 
ten as 

^2C^)_1^1 1 \_ 1 



B\A A + BJ A{A + By 



,2e('?) _ 1 ,„„A , B\ 1 



/3'^ = 7v7log l + -r 



f 52 ^V A) B{A + B) 
Since for ^(g) = q^/p we have 

m 1 eiq) 



q^ p q 



the above equations imply that 



1 I f B\ 1 

■log 1 + 



pA[A + B) 52 ^V A) B{A + B)' 

This shows that B /A satisfies (4.2) and, therefore, B = xA. From the equa- 
tions above we get 

(4.5) rn(3^eiq)il-q) ^ 



A+B l+x 
and since 9{q) = (1 — ^/p)q^'iq), 

m^P^eiq) = P-^m^qmq) = ^^(^ " J^j, 

(4.6) ^ 

p—l x'^ 

p \ + X '-' 

Even though we established Lemma 2 by looking at the Crisanti-Sommers 
representation (1.4), we will use (1.3) to write V{(i) :=7^(/3, 0) in order to 
compare more easily with the bound (2.21), which is not written in the 
Crisanti-Sommers form. [Of course, it would be possible to write (2.21) in 
the Crisanti-Sommers form with extra work.] For k = 2 and gi = 0,52 = q-, 
(1.3) becomes 

2P(/3)= inf f6-l + -log--^ 

b,m,g\ m Ui 

(4.T) 

- \0gD2 - mpH{q) - PHe{l) - e{q))]. 
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Since D2 = b — /3^(^'(1) — C'il)) we can minimize over D2 instead of b and 
to simplify the notations we will simply make the change of variable 

Then, since Di = D2 — m(3'^^'{q), we have 
2V{(3) = inf f/32^'(l) - /3^C'{q) + b-l-logb 

b,m,q \ 

(4.8) 

- 1 log [l - \m(3''i'{q)^ - mP^9{q) - p\e{l) - e{q))) . 

The infimum is achieved on {m,q) as in (4.1) and b = 1/(1 — q). Using (4.4), 
we get 

2P(/3) =/3V(l) - P^eiq) + b-l - log5- -log(l -6)-^ 
(4.9) 

-p^ie{i)-e{q)). 

Proof of Theorem 4. Given (3j,j < 2 as in Theorem 4, let rrij and 
qj be the corresponding solutions of (4.1) and bj = 1/(1 — qj) so that 'P(/9j) 
is given by (4.9). Let us first consider the overlap -R1.2 ~ u for |n| < y/qxQ2- 
Define c by \u\ = c^qiq2- We are going to use the bound (2.22) for A; = 4 
with m = (0, "m, mi, 771,2, 1) for some m > that will be specified later and 

V 62 / \ u cq2 

\u cq2j \u q2 

With these parameters the right-hand side of (2.22) becomes U{m, c) +TZ 
for 

[7(777, c) := -f3l{e{i) - e{q^)) - (3lie{i) - e{q2)) + /3?(e'(i) - e'(gi)) 
+ /3i(C'(i)-e'(^2)) 
+ 61 - 1 - log 61 + 62 - 1 - log 62 + 1 + n. 



where 



and 



-{miPf0{qi)+m2Ple{q2))il-(f) 

- m{f3le{qi) + Pie{q2) + 2/3i(320{V^I^)y 



II = -^ logf 1 - lrmf3ie{qi){l - c^-') 

7771 V bi 

- — logfl - ^m2PU'iq2){l - C^-')) - -log(l - 7775) 
7772 V O2 J m 
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for 

h- — rr 

i<2 



^ ^b,-m,(5U'{q,){l-cP-^] 



Since 9{y/qiq2) = vHo^Wio^), Lemma 4.4 implies that 

l=-(— + —)^{l-cn-m(— + - 
\mi 1712/ V"ii 'Ti 



7712/ 



and 



II = -— log(l - 5(1 - d'-^)) - — log(l - S{1 - c^-i)) 
mi 1712 

log [ 1 — m[ 1 



m V \"ii m2/l — 5{l — c^ ^) 

If we define mo by 

(4.10) i = ± + ± 

mo W-i 1712 

then for m, = ttiq 

I + II=-('J- + ^)^-('^ + ^)log(l-5). 

Therefore, comparing with (4.9), for all c 

C/(mo,c)=2P(/3i) + 2P(/32). 
The derivative 



d{c) 



dU{m,c) 



1 / . , 1-5 ScP' 



dm 
If p — I X 

satisfies 



-1 
„=^„-;^lv"^'' + ^°^l-5(l-cP-i)+l-<5 

2 



,, cP-log(l + xcP-^) + xcP-^ 

mn V P 1 + X 



(4.11) (i(0) = (i(l) = and (i(c) < forO<c<l. 

The equality d{l) = is equivalent to (4.2) and the fact that d{c) < for 
< c < 1 is a consequence of the following. The derivative d'{c) is equal to 
if c = 0, c = 1 or if 

I + xc^-^ = {1 + x)cP-^ . 

Making a change of variable a = c^~^ gives 

l + xo(P-^)/(P-2) = (l+x)a. 
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The left-hand side is convex in a, the right-hand side is hnear and two 
sides are equal at a = 1. Therefore, there is at most one solution on the 
interval (0,1) and d'{c) = has at most one solution on (0,1). However, 
since d{0) = d{l) = 0, d'{c) = has exactly one solution on (0, 1). Thus, d{c) 
cannot change sign inside (0,1) which proves (4.11). For < |m| < ^/qiq2, 
fluctuating m near itiq proves that 

FNiQs) <linfU{m,c) + n<V{/3i) + Vi(32) +n. 

Therefore, by concentration of measure the overlap cannot take values u 
with < \u\ < ^/<llQ2■ Lemma 2 will finish the proof of Theorem 4 if we 
can show that the overlap of two systems at the same (3 cannot take values 
|n| > q = l — 1//3. This can be shown by the usual replica symmetry breaking 
as in [8], which we will explain here for completeness. For specificity, let us 
assume that u > q. We are going to use the bound (2.22) for k = 3 with 
m = (0, m/2, n/2, 1) for some n such that m <n<2 and 

^3-(s ")■ e-^(j I). Q^^c :: 

With this choice of parameters, (2.22) becomes Fiy{Qi.) < [/(n, a) + 7?. where 
U{n,a) = -/3^(6'(1) - 9{q)) - nP^{e{u) - e{q)) - mp^9{q) 
+ /32(e'(l)-e'(n))+a-l-loga 

1 a-nP\e{u)-e{q)) 

log 

n a 

_lj^ a-nPHe{u)-e{q))-m(3^e{q) 

First of all, if we take n = \ and 

a = ao = 6 + (i\e{u) - i'{q)) = -^ + P^^u) - i'{q)) 

1-q 

we get U{l,ao) = 2V{[3) by comparing with (4.8). Furthermore, 

(9C/(n,ao) 

n=l 



d{u) 



dn 



= -(i\e{u) - e{q)) +(l- -)p\i'{u) - i'{q)){l - q) 

, l fi\i'{u)-i'{q))[l-q) ,,^n2,,n. .,..,, .. 

+ --, n2cii ^n ^ ^°§^ + ^ KiW-i (g) (1 - q)) 

m I — mp^'t, [q)[l — q) 

and we will show that d{u) < for u > q. In order to prove this, we will 
notice that d{q) = and show that d'{u) < for u> q. Indeed, 

d» = mu) (-U + '^''il"! -/'<;»'!-■"', + "'^'''^^ - '>' 



1 + 0Hi'(u) - £'(«))(! - 9) 1 - m/32f (,)(1 - ,) 
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The last term can be simplified using the fact that the first equation in (4.1) 
is equivalent to 



l-m/32f(g)(l-g) 
and then some simple algebra gives that for u> q, d'{u) < if and only if 

I — u I — q 
Since two sides are equal for n = (7 it is enough to show that for u> q 

32 /-"f-- ^ 



P^'iu) < 



(1-m)2- 

If we consider the function 

1 



F{u) = (3'C'{u) + -(- ^ 

m\l — q- 



+ mq l — q + m[q — u] 

then it is easy to check that the critical point conditions (4.1) are equivalent 
to 

F(0) = F(g) = 0, / F{u)du = Q. 

Jo 

Therefore, 

F'{u) = P'eiu) - -. -^-7 ^ = 

(1 — q + m[q — u))'' 

has at least two solutions on [0,q]. But this equation can be rewritten as 

{P^"{u)r^/^ = l-q + m{q-u) 

and since the right-hand side is convex in u, this equation has at most two 
solutions. Therefore, for u> q, 



(/?2^"(u))-^/2 >i_q^jj^(^q_u)>l 
which implies 



u, 



x'^{u) (1 — li)2 

This finally proves that for u> qhy fluctuating parameter n around 1 we 
get that Fp^{Qi.) < 2'P(/3) +7^ and this completes the proof of Theorem 4. 

D 
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5. Proof of Theorem 2. Let us first recall the main steps in the proof 
of Theorem 1 for Q such that minrj > 1 when (1.12) becomes Fj^^Q^) < 
nV{f3) + TZ. The remark following the proof of Theorem 1 states that we 
use interpolation (2.3) with A; = 1, Gaussian random vectors (^q j, . ■ . j-^oi) 
with covariance Aq = /3^<5^ where Q^ = Q — (3~^I is defined in (3.11) and 
Gaussian random vectors (zj j, . . . , z"j) with covariance Ai = (51. Then for 
the function Lp[t) defined in (2.6), Theorem 5 gives 

(5.1) ^'{t) = -^(3\9{1) - 9{q)) - Rit) + 7^, 
where the remainder R{t) was defined in (2.9). For ^{q) = g^/2 

(5.2) R{t) = i/32 Y. n{R"' - qlf?)t. 

j,j'<n 

In order to show that the upper bound of Theorem 1 is sharp we need to 
control R{t) and also prove that the bound on (p{0) given in Lemma 1 is 
sharp. Since 

\j<ni<N / 

we have 



(^(0) = ^ElogEi / exp ^ ^ a{{4^^ + z^ ■) dX% 

■'^^ j<ni<N 

= f + ^Elog/ exp5:^a^-4,dA^. 



'^^ j<ni<N 

On the other hand, (2.11) gives, using Ai as a parameter as in (2.22), that 
is, writing A2 = Ai + Ai, 

^(0) < ^(Tr(QAi) + Tr(AiQ) + Tt{A^'Ao) - n -log\Ai\) +n 
(5.3) ^ 

= ^ + 2 (Tr(AiQ) + Tr(Ar^Ao) - n - log |^i|) + 7^. 

Therefore, in order to show that this bound is sharp, we need to prove the 
following. 



Lemma 4. We have 



lim lim — Elog / exp > (<t-',z-') dA 



*3= j<n 

infi(Tr(ylQ) + Tr(^-iAo)-n-log|A|), 



where iP = (z^^, . . . ,2:^^). 
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The statement of the lemma will be proved for any Q and Aq, but for 
specific choices in the above interpolation it is easy to check that the infimum 
of (5.3) is achieved on 



(5.4) 



n , 



^i = /3/and lim lim v?(0) = -(3/3 - 2 - log/3). 

e— ♦0A'"^oo 2 



Proof of Lemma 4. The upper bound is given by (5.3). To prove the 
lower bound we will first replace integration over the sphere by a Gaussian 
integral constrained to the small neighborhood of the sphere. Let z> be a 
standard Gaussian distribution on M . If we write p G M as p = scr for 
<T G Sn then a and s are independent and the law of a is X^. Denote by 7 
the law of s. Clearly, 7([1 — e, 1 + e]) — > 1 for any e > 0. Let us consider a set 

(5.5) Oe = {(pS • • • , P") : Vj <n,s,<-[l-e,l + e],{a\..., a^) G Q J. 

First of all, for (p\ . . . , p") G 0^, 



^((p'-,z^-)-(^^z^-)) 






Therefore, 



/ exp^(^^z^)dA^ 



>7([l-e,l + e])""exp -eViV^ 



'j<n 



/ / exp V(sjcr^,z^) 



dXN{(T^) • • • <iAAr(cr") d-f{si) • • • d'y{sn) 



:7([l-e,l + e])-"exp -e^/iV^ 



i<" 



/ exp5^(p^-,z^)dz."(p\...,p-) 



and 



^Elog/ exp^(cr^z^)(iA^>lElog / exp^(p?-,z^)dz/" - 7^. 

We can now replace the set fi^ by the set defined in terms of the overlaps 
of p^ , j < n. Let us consider a set 

1^5 = {(p\ . . . , p") : R{p',p'') G [gjjv - S, qjj> + 5] for j,j' < n}. 
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Then, clearly, if we choose 6 = 5{e) small enough then 0,$ C [7^ and, there- 
fore, 

i-Elog/ exp^(^^z■'■)ciA^>lElog/ exp^(p^,z^)dz." - 7^. 

Thus, we replaced integration over the sphere by a Gaussian integral. Given 
a set V C (M^)"- and a symmetric nxn matrix A such that ^ + / is positive 
definite we define 

$^(F):=lElog|exp[^(p^z^)-i Y. a^.^P^ ,f^')\ du\ 

\j<n j,j'<n / 

First of all, 

(5.6) $A((M^r) = F{A) := l{Tr{{A + ly^Ao) - log | A + /|), 

which is easy to show by decoupling the coordinates and proceeding as in 
(2.17). Using the definition of fi^, we have 

i-Elog / exp5](p^z^■)dz."<iTr(^Q) + cI.^((M^)")+7^ 

= ]^TT{AQ)+F{A)+n, 

which, of course, coincides with the upper bound by making the change of 
variable A — > A + I. Let ^o be a matrix that minimizes Th:{AQ)/2 + F{A). 
Such Aq exists and is unique since F{A) is convex in A by Holder's inequality, 
the set A + 1 >Q is convex and F{A) — > +oo when | A + I| ^0. The critical 
point condition for Aq is 

(5.7) ^(^Ti{{Aq + xB)Q) + F{Aq+xB)^ 

for any symmetric matrix B. Let us consider the sets 

(5.8) V+, = {R{pi , p^' ) > q,j: + 6} , F.-., = {R{p^ , p^' ) < 9,y - 6} 
for j,j' < n, so that 

We will show below that if Y is any one of the sets (5.8) then for some c> 

(5.10) ^A,{V) < 'J>Ao((]R^)") - c = FiAo) - c. 
This implies that 

(5.11) hm <^Ao{^5) = lim $Ao((M^)") = F(Ao). 

iV— >oo N—ioo 



= 

x=Q 



(5.9) 



t,N\n 
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Suppose this is not the case. Then for some positive c > and N large 
enough we would have 

(5.12) $A„(n,)<$A„((M^n-c. 

Then by concentration of measure (5.10) and (5.12) would imply that with 
probability exponentially close to one for V equal to $7^ or one of the sets 
in (5.8) 

<eM-N/L) I exp('^(p^z^) - \ Y. ^Af^^f^')] ^^"• 

J(R^)" \j<n j,j'<n ) 

Adding up all these inequalities and using (5.9) we arrive at a contradiction, 
proving (5.11), which in turn implies that 

i-Elog / exp^(/f>^',zJ)di." = iTr(Aog) + <l>Ao(^5) +7^ 



= iTr(AoQ) + F(ylo)+7^ 

and this will finishes the proof of the lemma. It remains only to prove (5.10). 
The proof is the same for all V-"t-, and V-~ , so we will only consider the case 
V = V~-, for j / /. On the set V~-, for any x > 0, 

and, therefore, if we consider a matrix B with two nonzero entries 5jj/ = hji^j = 
then 

'^AoiV-j,) < x{qjj> -6) + <^Ao+xb{V-j,) < x{qjj> -6)+ F{Aq + xB) 

= U{x) 

■=-x5-\ TV(AoQ) + \ T\-((Ao + xB)Q) + F{Aq + xB). 

We have C/(0) =F{Aq) and using (5.7), C/'(0) = -5. Therefore, by slightly 
increasing x we get (5.10) and this completes the proof of lemma. D 

Next, we need to learn how to control the remainder term (5.2). The 
general approach is the same as in the proof of the Parisi formula in [7] (and 
in [8] or [4]), but the a priori estimates are now performed on 2n coupled 
copies of the system and will be similar in spirit to Theorem 1. Let us 
consider two 2n x 2n block matrices 

^' = (qi gV ""'""^ ^'"(q' q\ 
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and let (zq j, . . . jZq") and {zl ^, . . . , zf^) be independent Gaussian vectors 
with covariances 

Ao = P^G^ and Ai = P^{G^ - G^) = p'^{l - q)I = pi 

correspondingly, independent for i < N . Consider 

(5.13) Ht{cT\...,cT'n = VtY.(^H,,{a^) + VT^tY: E^K4^ + <.) 

j<2n j<2ni<N 

and 

(5.14) h{cT\...,CT'-) = VtJ2(^Hr,{a^)+Vl^tY: E^Hr 

j<2n j<2ni<N 

The Gibbs average {■)t in (5.2) can be rewritten as 

{f{a\...,a^^))t= f fexphtdXJ^/ f exphdXJ^ 

and the remainder (5.2) can be rewritten as 

(5.15) R{t) = \(3'' E E n{R3,r - 9h'f)t- 

Given a set A C S"^ , consider 



^^" 



^{A,t) = ^ElogEi / expHtdXJ 
(5.16) "^ ^^ ^ 

A* Ja 

Consider n x n matrix E = {cjj' ) with elements Cjj' £ [—1,1] such that 



q2 e 

E Q- 



U-^ ^ ^2 



is a nonnegative definite matrix and consider the set 

(5.17) Ue = {{c7^,...,(T^'')£Sp: Rjj> G [ujj> - e, Ujj> + e] for j, / < 2n} . 

Let us define 

m = f (3/3 - 2 - log/3 - (3H{eil) - 9{q))) 

(5.18) 

= -(3/3-2-log/3-t(/3-l/2)). 

Theorem 2 will then follow from the following a priori estimate. 
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Theorem 6. For any to < 1 ('''^-d for any t <tQ, for N large enough, 

1 
K 



(5.19) ^iUe,t) < 2V(t) - ^ E (^iJ' - 4')' + ^ 



j,j'<n 

where a constant K does not depend on N,t and U. 
Let us first show how Theorem 6 imphes Theorem 2. 
Proof of Theorem 2. First of all, 

(5.20) V'(i) = ^ (2/? - ^ - log/?) = ^np)- 

Let us take K as in (5.19) and for ei > let us define a set 

V = \e= (e,,,v) G [-1, 1]"' : ^ (e,y - q]j,f > 2i^(V(t) - v?(t)) + 2KeX 

^ j,j'<n ) 

For any E £V, (5.19) implies that 

(5.21) ^([/e, t) < 2-0(0 - ^{2K{iP{t) - ifit)) + 2Kei) + 7^ < 2^{t) - ei 

K 

for large enough N and small enough e. Everywhere below let L denote a 
constant that might depend on ei and K denote a constant independent of 
ei. Since 

2v9(t) = ^(Q, xQ„t), 

(5.21) and concentration of measure imply that 

(5.22) ¥.{I{Ue))t <Lexp{-N/L) 

and the constant L here does not depend on E. Let us consider the set 

= 1 E E {Rj,f-9lr?>2K{m-^{i))+^KeAn{Q,xQ,). 

il<j<nn<j'<2n ) 

We can choose a sequence Ei£V for i < K[e) and large enough K[e) such 
that 

OC (J Ue{Ei), 
i<K(e) 

where we made the dependence of Ue on E explicit. Inequality (5.22) implies 
that 

(5.23) E(/(0))i < LK{e) exp{-N/L). 
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Using the definition of tp in (5.18) and (5.1), 
(5.24) {ij{t)-^{t))' = R{t)+n. 

On the complement W of we have 

E E iRj,r-9lff<2K{m-V>it)) + 2Ks,, 

l<j<nn<j'<2n 

and (5.23) impUes that 

R{t) < K{{i^{t) - v?(t)) + £1 + E{I{n))t) 

< K{iP{t) - ip{t)) + Kei + LK{e) exp{-N/L). 

The relation (5.24) now implies that 

(V(t) - ^(t))' < Kiijit) - ^(t)) + Kei + LK{e) exp{-N/L) + 7^. 

Since by Lemma 4 we have linig^o hniTv^oo <^(0) = V'(O)) solving this differ- 
ential inequality and then letting A^ — > oo, e — > and ei — > implies that 

lim lim ip(t) = ip{t) for t < t^. 
e—*0 N~^oo 



Since the derivatives il^'{t) and (p'{t) are both bounded we have 
Using (5.20), 



lim sup lim sup 1 99(1) — ip{l)\ < K{1 — to). 



limsuplimsup \ip{l) — nV{(3)\ < K{1 — to) 

and letting to — > 1 completes the proof of Theorem 2. D 

Proof of Theorem 6. Let U'^ = U and let U^ be a symmetric non- 
negative definite matrix that will be specified later such that C/^ — U^ is also 
nonnegative definite. Let 

A'o = /32[/i and A[ = p\U^-U^). 

Let (yQ j, . . . , y'ol) and (y| j, . . . , yfj) be Gaussian vectors with covariances 
Aq and A']^ correspondingly, independent for i < N. For < s < 1, consider 

Hs{ct\. . . ,a^^) = V7t Y, (3HNicT^) 

j<2n 

(5.25) + VT^sVi ^ ^ alivl, + y{j 

j<2ni<7V 
j<2ni<Ar 
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and let 



—E log El / expHsdX 



2n 

TV ■ 



By a straightforward computation as in Theorem 5 one can show that 

2 






ij'<2n 

where the remainder R{s) > 0. The analogue of the first line in (2.7) is not 
present here because U"^ = U, that is, the covariance parameters match the 
constraints on the overlaps. Therefore, 

(5.26) M/([/„t) = ,^(i)<</,(o)-^ Y. {0{uly)-eiuy)) + n. 

j,j'<2n 

Note that 

j<2ni<N 

where xj ^ = Vtyj^ + \/l — te/j and, therefore, the vectors (xj j, . . . , Xq") and 
{x\ j, . . . , xfl) have covariances 

Aq = (1 - t)Ao + tA'o and A* = (1 - t)Ai + tA[. 

Lemma 1 now implies that for any 2n x 2n symmetric positive definite matrix 
A such that Ai = A — A\ is also positive definite we have 

2(/.(0) < Tt{A\U) + Tt{AiU) -2n + Tt{A^^A^q) - log \Ai\+TZ 

and combining with (5.26) 

2^(C/£, t) < Tt{A\U) + TiiAiU) -2n + Tr(^^^A^) - log |^i| 
(5.27) 

-t/?2 J2 i0iul,,)-eiul^,)) + n. 

j,j'<2n 

We proceed by a diagonalization procedure as in the proof of Theorem 1. 
Let C/^ = O^ B?0 for R^ = Diag(r^, . . . , r^^) and orthogonal matrix O. Take 
R^ = Diag(rj, . . . , r^n) > such that R? — R^ is positive definite and let U^ = 
O^R^O. Take B = Diag(6i, . . . , ftsn) and let Ai = O'^BO. Since Ai = p^{l - 
q)I, we have Ai = O^AiO. Let C be a matrix such that Aq = P^O^CO. 
Then 

A* = 13^0'^ {{1 - t)C + tR^)0 

and 

A{ = /320^((i - i)(l - q)I + t{R^ - R^))0. 
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Since for e{q) = q^ /2 



Jj'<2n j<2n. 



the bound (5.27) becomes 



2*([/„t) < Y. /32(t(r| - r]) + (1 - t)(l - g))r| + 6,rf - 1 

j<2n ^ 

(5.28) + ^/32(ir] + (1 - t)c,,j) - log 6,- - t(i\e{T]) - 0(r]))) 



+ 7^. 



We will take 



(5-29) h = -JT—I 

j J 

and the bound (5.28) becomes 

mUs,t) < E (IP'tiir^f - ir]f) + ^ - 1 

(5.30) + /32(1 - t)((l - g)r2 + (rj - r])c,-,) + log(r| - r]) 

+ 7^. 



Let 



(5.31) 






In the first case the jth term in the sum (5.30) becomes 

Mr]) = Ptr] -i+(]r]-l + p\l- t)(l - q)r] + /3(1 - t)cj,, - log/? 

= f3tr] -i + (3r]-l + (3{1 - t)r] + /3(1 - t)c,-,- - log/3 
since 1 — g = 1//3, and in the second case it becomes 

/2(r|) = ^{r]f + /?(! - t)r2 + p\l - t)r]c,,, + logr]. 
It is easy to check that 

/i(r') = /2(r') and f!,i(3-')-f[ir') = (3\l-t)cj,,>0 
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since C is nonnegative definite. Also, 

/2 (-1) =P't-J^< -/3'(1 - *) < for rf < i 

and fi{rj) = and, therefore, 

1 



(5.32) /2(r|)</i(r|) for r| < 



Therefore, 

(5.33) 2*(f/„t)< ^/l(r|) + 7^ 

j<2n 

and this bound is achieved by taking r^ = r'^ — 1/(3 in (5.30). It is important 



to note that in this bound we no longer have to assume that r'j > 1/(3 and 
even though for r| < 1/(3 parameter '''] = '''] — ^/P becomes negative, due to 
(5.32) we can simply treat (5.33) as a formula. Since 

Y^ r| = Tr(i?2) = Tr([/2) = 2n and ^ Cjj = Tt{C) = Tr(G^) = 2nq 

j<2n j<2n 

we have 

Y^ f^{r]) = 2n(3t -nt + 2n(3 -2n + 2n(3{l -t) + 2n(3{l - t)q - 2n\og(3 

j<2n 

= 2n(3/3 - 2 - log /3) - 2nt{(3 - 1/2) = Ai(:{t) 

by comparing with (5.18). Therefore, (5.33) implies that 

(5.34) ^{Ue,t)<2ip{t)+TZ. 

However, we can improve upon (5.34) by slight fluctuations of Ai in (5.27). 
Our choice of r^ = fj — ^/P and (5.29) imply that bj = (3, that is, B = (31 
and, therefore, in (5.27) we have Ai = (31. Also, R^ — R^ = (3~^I and, hence, 
[/I = C/2 _ ^-ij^ Lg^. ^g ^.j^j^g i-j^g derivative of (5.27) in Ai at Ai = (31. If Ai 
was not constrained to be symmetric, we would have 

d 
—— r.h.s. of (5.27) 

= {U- (A^r - {Aj'AiA^r) , =U'- i3-'l - r^A* 

A\=lil 

= \J^-(1- t)G^ - tU^ = (1 - t){G^ - [/!), 

where we used some well-known formulas for derivatives of matrix deter- 
minants and inverses. However, since Ai = {aj.j') is symmetric, Oj j/ = dj'j, 
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the derivative in the off-diagonal element will simply be doubled, that is, for 

^^•.^•' = 7^ '■^■'- °f (5-27) , _,, = 2(1 - t)iG' - U\,,. 



daj,r 



Ai=pi 



Since 

^( Q^-E\ 

\Q^-E j' 

for 1 <j <n and n< f < 2n we have 

^ij' = 2(1 - t){g]j, - u]j,) = 2(1 - t){q]j,_^ - ejj/_„). 

Also, since the second derivatives in aj ji are bounded in some neighborhood 
of Ai = pi, this implies that 

(5.35) ^{Ue,t) < 2^{t) -^ J2 (lh' - e^')' + ^ 

j,j'<n 

and this completes the proof of Theorem 6. D 
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